Apparatus and Methods for Fluorescence Imaging Using Radiofrequency-Multiplexed Excitation

ABSTRACT

Apparatus and methods for fluorescence imaging using radiofrequency multiplexed excitation. One apparatus splits an excitation laser beam into two arms of a Mach-Zehnder interferometer. The light in the first beam is frequency shifted by an acousto-optic deflector, which is driven by a phase-engineered radiofrequency comb designed to minimize peak-to-average power ratio. This RF comb generates multiple deflected optical beams possessing a range of output angles and frequency shifts. The second beam is shifted in frequency using an acousto-optic frequency shifter. After combining at a second beam splitter, the two beams are focused to a line on the sample using a conventional laser scanning microscope lens system. The acousto-optic deflectors frequency-encode the simultaneous excitation of an entire row of pixels, which enables detection and de-multiplexing of fluorescence images using a single photomultiplier tube and digital phase-coherent signal recovery techniques.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/792,282 filed on Jul. 6, 2015, incorporated herein by reference inits entirety, which is a 35 U.S.C. § 111(a) continuation of PCTinternational application number PCT/US2014/010928 filed on Jan. 9,2014, incorporated herein by reference in its entirety, which claimspriority to, and the benefit of, U.S. provisional patent applicationSer. No. 61/750,599 filed on Jan. 9, 2013, incorporated herein byreference in its entirety. Priority is claimed to each of the foregoingapplications.

The above-referenced PCT international application was published as PCTInternational Publication No. WO 2014/110290 on Jul. 17, 2014, whichpublication is incorporated herein by reference in its entirety.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

This invention was made with Government Support under W81XWH-10-1-0518,awarded by the U.S. Army, Medical Research and Materiel Command. TheGovernment has certain rights in the invention. Not Applicable

INCORPORATION-BY-REFERENCE OF MATERIAL SUBMITTED IN A COMPUTER PROGRAMAPPENDIX

Not Applicable

BACKGROUND OF THE INVENTION 1. Field of the Invention

This invention pertains generally to optical imaging devices and methodsand more particularly to apparatus and methods for high-speed, one andtwo-dimensional fluorescence imaging enabled by beat frequencymultiplexing.

2. Description of Related Art

Fluorescence microscopy is one of the most important, pervasive andpowerful imaging modalities in biomedical research. The spatialresolution of modern fluorescence microscopy has been improved to such apoint that even sub-diffraction limited resolution is routinelypossible. However, time resolution in fluorescence microscopy has notkept pace with advances in spatial resolution.

While a number of fluorescence microscopy modalities exist, the timeresolution of the technique is fundamentally limited by the relativelyweak optical emission of fluorescent samples. As a result, the maximumfull-frame (512×512 pixels) rate of traditional single-point laserscanning fluorescence microscopy is limited to approximately a videorate of 30 frames per second, which corresponds to pixel rates of lessthan 10 MHz. Linescan and spinning disk confocal microscopes are capableof higher frame rates by multiplexing the fluorescence excitation anddetection, but the frame rates are ultimately limited by the lowelectronic gain and both the optical throughput of the spinning disk andread-out time of the detector, respectively.

The demand for sub-millisecond time resolution in fluorescencemicroscopy has been the primary driving force behind the development ofmany advanced imaging technologies, such as the electron multipliercharge coupled device (EMCCD) camera, the scientific complementarymetal-oxide-semiconductor (sCMOS) camera, the Nipkow spinning diskconfocal microscope, and the linescan confocal microscope. While each ofthese devices present distinct advantages and tradeoffs betweensensitivity, speed, resolution, and confocality, a device for imagingthe sub-millisecond biochemical dynamics in live cells and in vivoremains an outstanding technical challenge.

High throughput imaging flow cytometry is another application in whichhigh speed fluorescence imaging is required. Imaging of individual cellsin a fluid flow, compared with measuring only scattering and singlepoint fluorescence amplitudes, provides information that can be utilizedfor high-throughput rare cell detection, as well as morphology,translocation, and cell signaling analysis of a large number of cells ina short period of time. The high flow rates associated with flowcytometry demand high sensitivity photodetection and fast camera shutterspeeds to generate high SNR images without blurring.

Conventional imaging flow cytometers use time delay and integration CCDtechniques in order to circumvent this issue, but the serial readoutstrategy of this approach limits the device to a throughput of 5,000cells per second. At this rate, high efficacy detection of rare cellsusing flow cytometry, such as circulating tumor cells in blood, is notpractical for clinical applications.

The tradeoff between speed and sensitivity is a significant limitingfactor in high-speed fluorescence microscopy systems. The ability togenerate a high signal to noise ratio (SNR) image from the small numberof photons emitted from a fluorescent sample during a short(sub-millisecond) time period traditionally relies on the ability of thephotodetection device to provide electronic gain such that the detectedsignal is amplified above its thermal noise level. For this reason, highgain photomultiplier tubes (PMT) and EMCCDs are used most frequently forhigh-speed fluorescence imaging applications. While modern EMCCDsexhibit high quantum efficiency and gain, the gain register and pixelreadout strategy is serial, which limits its overall full frame rate tofewer than 100 frames per second. PMT's offer higher gain, lower darknoise, and higher readout speed than EMCCDs, but are typically notmanufactured in large array formats, limiting their utility tosingle-point-scanning applications. Due to the use of a PMT detector,laser scanning fluorescence microscopy is capable of high sensitivity atsimilar frame rates to EMCCDs, but the serial nature of the beamscanning ultimately limits the speed of image acquisition.

To date, these technological shortcomings have prevented full-framefluorescence imaging analysis of sub-millisecond phenomena in biology.There is a need for an imaging device that can resolve the subtledynamics of biochemical phenomena such as calcium and metabolic waves inlive cells, action potential sequences in large groups of neurons, orcalcium release correlations and signaling in cardiac myocytes.

Accordingly, there is need for an apparatus and method for fluorescencedetection and imaging that is fast with sub-millisecond time resolutionto capture dynamic processes in cells as well as flow imaging that canquickly perform high-throughput morphology, translocation and cellsignaling analysis on large populations of cells. The present inventionsatisfies these needs as well as others and is an improvement in theart.

BRIEF SUMMARY OF THE INVENTION

The present invention provides a new approach to fluorescencestimulation, detection and imaging using radiofrequency-multiplexedexcitation (FIRE) laser scanning microscopy. The FIRE system employsorthogonal frequency-domain multiplexing techniques to provide bothparallel pixel excitation and nominal pixel readout rates ofapproximately 80 MHz. However, pixel frequencies in the range of0.1-1,000 MHz are possible.

Acousto-optic deflectors are used to frequency-encode the simultaneousexcitation of an entire row of pixels, which enables detection andde-multiplexing of fluorescence images using a single photomultipliertube and digital phase-coherent signal recovery techniques.

Specifically, FIRE microscopy uses radiofrequency-to-space mapping toencode the excitation of a fluorescent sample into the frequency domain,such that the image can be detected using a single PMT, and demodulatedusing digital lock-in detection.

In one embodiment, the FIRE system is adapted to provide a scalableapproach to high-speed imaging, in that it leverages the MHz bandwidthsavailable to radiofrequency (RF) electronics and acousto-optic devicesto enable MHz-equivalent pixel clock readout of hundreds of pixelssimultaneously in a single data stream. Additionally, FIRE microscopyuses direct digital synthesis (DDS) to engineer the amplitude and phaseof each individual pixel excitation frequency, which enables applicationof phase-coherent digital lock-in amplifier and orthogonalfrequency-division multiplexing (OFDM) algorithms to the imagede-multiplexing process in order to reduce image noise and pixelcrosstalk. The system flexibility afforded by the use of direct digitalsynthesis enables full adjustment of the pixel number, readout rate, andfield of view.

The apparatus is configured to produce frequency shifted light beamsthat can be used to interrogate multiple points on a samplesimultaneously such that each individual point of the sample is at adistinct radiofrequency. To accomplish this, an excitation laser beam isfirst split into two arms of a Mach-Zehnder interferometer. The light inthe first arm is frequency shifted by an acousto-optic deflector, whichis driven by a phase-engineered radiofrequency comb that is designed tominimize the signal's peak-to-average power ratio. This RF combgenerates multiple deflected optical beams possessing a range of bothoutput angles as well as frequency shifts. The second arm of theinterferometer is shifted in frequency using an acousto-optic frequencyshifter to produce a local oscillator (LO) beam. A cylindrical lens maybe used to match the angular divergence of the LO arm to that of the RFcomb beams. After combining at a second beam splitter, the two beams arefocused to a line on the sample using a conventional laser scanningmicroscope lens system. High-speed line scanning of the sample isaccomplished using a resonant scan mirror in the transverse direction.

Fluorescent molecules in the sample function as square-law detectors, inthat their excitation responds to the square of the total electricfield. The resulting fluorescence is emitted at the various beatsdefined by the difference in frequencies of the two arms of theinterferometer. The fluorescence emission thus oscillates at theexcitation frequency with an appreciable modulation, given an excitationfrequency that is not much greater than 1/τ, where i is the fluorescencelifetime of the sample. For fluorophores with lifetimes in the singlenanosecond range, the useable RF bandwidth of FIRE is approximately 1GHz. Since acousto-optic devices are inherently resonant, the frequencyshifter in the second arm of the interferometer is chosen to heterodynethe beat frequencies to baseband in order to maximize the useablebandwidth for a given fluorophore.

Fluorescence emission from the sample is preferably collected by theobjective lens, and is detected by a PMT in a de-scanned confocalconfiguration, using a slit aperture to reject fluorescence emissionfrom other sample planes.

As a demonstration of the technique, diffraction-limited confocalfluorescence imaging of stationary cells at a frame rate of 4.4 kHz, andfluorescence microscopy in flow at a velocity of 1 m s⁻¹, correspondingto a throughput of approximately 50,000 cells per second was performed.

In combination with fast fluorescent indicators and voltage-sensitivedyes, this high speed imaging modality has the potential to observepreviously temporally-unresolved sub-millisecond dynamics in biology,which may lead to a more complete understanding of neural function,autoimmune diseases, cardiac arrhythmias, and othermillisecond-timescale biological phenomena.

Beat frequency multiplexing is also applicable to other types of laserscanning microscopy, including two-photon excited fluorescencemicroscopy. Perhaps most notably, because emission from each pixel istagged with a distinct radiofrequency, FIRE is inherently immune topixel crosstalk arising from fluorescence emission scattering in thesample—the effect that typically limits the imaging depth in multifocalmulti-photon microscopy. In combination with fast fluorophores, FIREmicroscopy is capable of observing nano- to microsecond timescalephenomena using fluorescence microscopy.

Further aspects of the invention will be brought out in the followingportions of the specification, wherein the detailed description is forthe purpose of fully disclosing preferred embodiments of the inventionwithout placing limitations thereon.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

The invention will be more fully understood by reference to thefollowing drawings which are for illustrative purposes only:

FIG. 1 is a schematic diagram of one embodiment of an apparatus forfluorescence imaging according to the invention.

FIG. 2A is a schematic diagram of an acousto-optic deflector element ofFIG. 1 that produces a single diffracted first-order beam for eachradiofrequency comb frequency.

FIG. 2B is a schematic diagram of a beam splitter showing beat frequencygeneration from mixed frequency shifted beams.

FIG. 3 is a flow diagram of one method for inducing and detectingfluorescence from a sample with frequency shifted beams according to theinvention.

FIG. 4 is a flow diagram of one method for fluorescence microscopyaccording to the invention.

FIG. 5 is a flow diagram of one image reconstruction algorithm accordingto the invention.

FIG. 6 is a flow diagram of a second image reconstruction algorithmaccording to the invention.

FIG. 7 is a flow diagram of a third image reconstruction algorithmaccording to the invention.

DETAILED DESCRIPTION OF THE INVENTION

Referring more specifically to the drawings, for illustrative purposesseveral embodiments of the system scheme of the present invention andthe associated methods for fluorescence excitation and detection aredepicted generally in FIG. 1 through FIG. 7 . It will be appreciatedthat the methods may vary as to the specific steps and sequence and theapparatus architecture may vary as to structural details, withoutdeparting from the basic concepts as disclosed herein. The method stepsare merely exemplary of the order that these steps may occur. The stepsmay occur in any order that is desired, such that it still performs thegoals of the claimed invention.

Turning now to FIG. 1 , FIG. 2A and FIG. 2B, one embodiment of anapparatus 10 for fluorescence imaging using radiofrequency multiplexedexcitation is schematically shown to illustrate the invention. Excitingindividual points on the sample at a distinct radiofrequency is animportant feature of FIRE apparatus and procedure. In the embodimentshown in FIG. 1 , beat frequency excitation multiplexing is performed byusing acousto-optic devices in a Mach-Zehnder interferometer (MZI)configuration.

An excitation laser 12 produces a beam that is split by a non-polarizingbeam splitter 14 and the resulting first beam 16 is directed to anacousto-optical deflector (AOD) 22 and the second beam 18 from thesplitter 14 is directed to an acousto-optic frequency shifter (AOFS) 28.

As shown in FIG. 1 , the excitation light beam 16 in one arm of the

MZI can be frequency shifted by a 100 MHz bandwidth AOD 22, for example,driven by a comb of radiofrequencies produced by the DDS RF combgenerator 20. The comb of radiofrequencies from the DDS RF combgenerator 20 is preferably phase-engineered to minimize its peak-toaverage power ratio. As seen in FIG. 2A, the AOD 22 produces multipledeflected optical beams 24 with a range of both output angles andfrequency shifts. The AOD 22 produces a single diffracted first-orderbeam for each radiofrequency comb frequency.

Light beam 18 in the second arm of the interferometer passes through anacousto-optic frequency shifter 28, preferably driven by a singleradiofrequency tone produced by a DDS RF tone generator 26, whichprovides a local oscillator (LO) beam 30. A cylindrical lens (not shown)may be placed after the AOFS 28 to match the divergence of the localoscillator (LO) beam 30 to that of the radiofrequency beams.

At the MZI output, the two beams 24, 30 are combined by a second beamsplitter 32. FIG. 2B also shows the beat frequency generation at the MZIoutput and beam splitter 32. The combined beam 34 is ultimately focusedto a horizontal line on the sample, mapping frequency shift to spaceusing a conventional laser scanning microscope lens system.

In the apparatus of FIG. 1 , the combined beam 34 is reflected fromdichroic mirror 36 to a resonant scan mirror galvanometer 38 to thesample through the scan lens 40, tube lens 42 and objective lens 44. Thesample cells 46 are in a channel 48 and are brought through the combinedbeam 34 in this illustration.

Fluorescence emissions 50 from the exposed sample 46 are detected by aphotomultiplier tube (PMT) 58 in a confocal configuration, using a slitaperture 56 to reject out-of plane fluorescence emission. A resonantscan mirror 38 performs high-speed scanning in the transverse directionfor two-dimensional imaging. Points in the horizontal direction areexcited in parallel at distinct radio frequencies. Scanning this linescan excitation in the vertical direction using a galvanometer generatesa two-dimensional image.

In the embodiment of FIG. 1 , the emission beam 50 from the objectivelens 44, tube lens 42, scan lens 40 and resonant scanning mirror 38 isdirected through the dichroic mirror 36 and fluorescence emissionfilters 52, 54 and slit aperture 56 to at least one photomultiplier tube58. The photomultiplier tube 58 is coupled to a computation andrecording device 60 such as a digital recording oscilloscope.

In one embodiment, the computation device 60 is configured to controlthe other components such as the resonance scan mirror 38, laser 12, RFcomb 20 and RF tone 26 generators, and optics as well as process the PMTsignals. The computation device may also have a display.

Accordingly, a method for fluorescence excitation and detection isprovided utilizing frequency shifted light that can be used tointerrogate multiple points on a sample simultaneously. One method 70for fluorescence excitation and detection is shown schematically in FIG.3 . At block 72 of FIG. 3 , frequency shifted beams of light arecreated. Frequency shifted beams of light can be created by differentmethods. A single diffracted first order beam for each radio frequencycomb frequency can be produced with an acousto-optic deflector as shownin FIG. 2A.

Because fluorescent molecules in the sample 46 function as square-lawdetectors of the total optical field, fluorescence is excited at thevarious beats defined by the difference in frequencies of the two armsof the interferometer. Given the finite frequency response offluorophores, the LO beam frequency shift is chosen to heterodyne thebeat frequency excitation spectrum to the baseband to maximize theusable modulation bandwidth. This is necessary because AODs typicallyoperate over an upshifted, sub-octave passband to avoid harmonicinterference.

Direct digital synthesis (DDS) of the radiofrequency comb 20 used todrive the AOD 22 defines each pixel's excitation by a specificradiofrequency and phase, resulting in phase coherence between theradiofrequency comb and the detected signal. This phase coherenceenables image de-multiplexing using a parallel array of phase-sensitivedigital lock-in amplifiers, implemented in the computer 60. FIRE'sparallel readout results in a maximum pixel rate equal to the bandwidthof the AOD.

Since each pixel's excitation is defined by a specific radiofrequencyand phase using the direct digital synthesizer (DDS), the detectedfluorescence is phase-shifted and scaled with respect to the drivingsignal. This phase-coherent emission enables digital lock-in amplifiersignal recovery, a well known ultra-sensitive detection technique.

At block 74 of FIG. 3 , a sample is interrogated with the frequencyshifted beams with each point at a distinct radio frequency to excitefluorescence in the sample. The response of the sample to exposure tothe frequency shifted beams is detected at block 76 of FIG. 3 .

A significant feature of the FIRE apparatus and methods is its abilityto excite fluorescence in each individual point of the sample at adistinct radiofrequency. Digitally synthesized radiofrequency “tagging”of the fluorescence emission of each pixel occurs at the beat frequencybetween two interfering frequency-shifted laser beams. With frequencydomain multiplexing, each pixel in a row of a FIRE image is assigned itsown radiofrequency. In a two-dimensional FIRE image, pixels areanalogous to points on a time-frequency Gabor lattice. A single-elementphotodetector simultaneously detects fluorescence from multiple pixels,and an image is reconstructed from the frequency components of thedetector output, which are resolved using parallel lock-in amplificationin the digital domain. The response of the sample to exposure to thefrequency shifted beams is detected at block 76 of FIG. 3 . The analysisof the detected signal may be simply a multi-point measurement of thesample or may include the creation of an image of the subject.

One adaptation of the method for fluorescence microscopy 80 is shownschematically in FIG. 4 . At block 82, a frequency shift of a firstlight beam is performed with an acousto-optic deflector driven by a combof radiofrequencies, preferably phase-engineered to minimize its peak-toaverage power ratio. The acousto-optic deflector (AOD) produces multipledeflected optical beams with a range of both output angles and frequencyshifts.

Light from a second beam passes through an acousto-optic frequencyshifter, driven by a single radiofrequency tone, which provides a localoscillator (LO) beam at block 84 of FIG. 4 . An optional cylindricallens may be used to match the LO beam's angular divergence to that ofthe radiofrequency comb beams.

Interrogation of the sample with the beams at block 86 preferably uses apixel multiplexing approach using beat frequency modulation. In oneembodiment, the frequency shifted beams are directed to a sample using aconventional line scanning microscope lens system. High-speed linescanning of the sample can be accomplished using a resonant scan mirrorin the transverse direction.

Since the excitation of the fluorescent molecules in the sample respondsto the square of the total electric field, the resulting fluorescence isemitted at the various beats defined by the difference frequencies ofthe first and second interrogating beams:

E _(t) =E _(RF) +E _(LO)

|E _(t)|² =E* _(t) E _(t)=2E ₀ ²{1+cos(ω_(RF) t)}

Interference of two frequency shifted waves results in amplitudemodulation of the resulting intensity at the beat frequency:

E _(RF) =E ₀ e ^(j(ω) ^(o) ^(+ω) ^(RF) ^()t)

E _(LO) =E ₀ e ^(jω) ⁰ ^(t)

The fluorescence emission thus oscillates at the excitation frequencywith an appreciable modulation, given an excitation frequency not muchgreater than lit, where ti is the fluorescence lifetime of the sample.

The maximum modulation frequency, and thus maximum pixel readout rate,is intrinsically limited by the sample's fluorescence lifetime. If theexcitation frequency is less than lit, where i is the fluorescencelifetime of the sample, the emitted fluorescence will oscillate at theexcitation frequency with appreciable modulation.

Furthermore, beat-frequency modulation is fundamental to the speed ofFIRE; the beating of two coherent, frequency-shifted optical wavesproduces a single radiofrequency tone, without any harmonics that canintroduce pixel crosstalk and reduce the usable bandwidth. Because theFIRE system is designed so that every pixel is spatially resolved at thediffraction limit, beating of the excitation light from two adjacentpixels generates a fluorescence signal at the comb spacing frequency.

However, this frequency lies out of the image band and does not producepixel crosstalk. With respect to blur-free imaging of fast phenomena insamples such as living cells, the speed of the fastest dynamic eventthat can be imaged is determined by the maximum allowable sidebandfrequency (half the comb spacing).

The spatial resolution, number of pixels and field of view can also bedesigned. The FIRE microscope is a diffraction-limited technique. Likeother laser scanning microscopy techniques, the minimum transversespatial resolution is the diffraction limited spot size determined bythe numerical aperture of the objective and the laser excitationwavelength.

The preferred pixel rate of the system is equal to the product of thenominal line rate and the number of pixels per line. With the nominalline rate limited to the comb spacing, the maximum pixel rate of FIRE isequal to the bandwidth of the AOD (B_(AOD)), givenB_(AOD)=p_(x)×r_(line), where p_(x) is the number of pixels in a line,and r_(line) is the line rate. The x-axis was chosen as the AODdeflection direction.

The number of resolvable points per line is equal to the time-bandwidthproduct of the AOD (TBP_(AOD)). To satisfy the Nyquist spatial samplingcriterion, the number of pixels per line should be at least twice thisvalue, P_(x)=2×TBP_(AOD).

The field of view in the x-direction (FOV_(x)) is the number ofresolvable points times the diffraction-limited spot-size, d: givenFOV_(x)=d×TBP_(AOD)=d×p_(x)/2.

Nyquist oversampling in the y-direction should also be satisfied. In thecase of 2-D imaging with a scan mirror, the frame-rate, field of viewand number of pixels in the y-direction should satisfy the relationsp_(y)×r_(frame)=r_(line) and FOV_(y)=d×p_(y)/2, where r_(frame) is the2-D frame rate. The range of the scan mirror should be chosen to matchthe FOV_(y) above.

These criteria for oversampling in the y-direction are not unique toFIRE, and can be satisfied for any imaging system. However, satisfyingthe Nyquist criterion in the y-direction is particularly preferred forFIRE. Under-sampling in the y-direction can lead to a rate of change offluorescence that is greater than half the line-rate, even for staticsamples. In the frequency domain, the generated sidebands will extendbeyond half the frequency comb spacing, resulting in inter-pixel crosstalk and blurring in the x-direction.

With oversampling of each diffraction limited spot of a sample, thisblurring effect can be avoided. For dynamic samples, such as live cells,the frame rate and pixel frequency spacing should also be chosenappropriately to avoid sideband blurring, depending on the sampledynamics. The pixel spacing in frequency space must be at least twice aslarge as the maximum rate of change of the fluorescence signal. Theoptical system magnification should be adjusted such that individualpixels are spatially resolved at the sample.

In the FIRE system described above, the fastest line scan shutter speedis 1.25 ms (800 kHz comb spacing), which allows for the capture of pixeldynamics with frequency content up to 400 kHz.

Fluorescence from the interrogated sample is detected over time at block88 and an image can be generated at block 90 of FIG. 4 . Detection ispreferably performed by at least one photomultiplier tube (PMT).However, other detection schemes may be adapted.

Fluorescence imaging generates darkfield images of samples by estimatingthe number of emitted photons from each excited pixel of a sample. Inconventional fluorescence imaging, the number of photons is estimatedthrough integration of the optical signal. In FIRE, the light emittedfrom each pixel is modulated at a particular radiofrequency, and thestrength of the signal coming from each pixel is differentiated eitherby performing a short time Fourier transform, or demodulating the signalfrom each pixel using parallel digital lock-in amplification, forexample. As a result, FIRE reads out multiple pixels in parallel,increasing imaging speed and increasing the integration time for a givenframe rate.

The flexibility afforded by digitally synthesizing the amplitude andphase of the radiofrequency spectrum provides complete, real-timecontrol over the number of pixels, pixel frequency spacing, pixelnon-uniformity and field of view. Because PMTs inherently have a smallerdynamic range than CCD or CMOS technologies, maximizing this quantityper pixel is important to the performance of the FIRE procedure.Specifically, phase-engineering the excitation frequency comb enablesthe dynamic range of each pixel to scale as D/√M, where D is the dynamicrange of the PMT, and M is the pixel-multiplexing factor. This is incontrast to the case where the initial phases of all of the excitationfrequencies are locked, which yields images with a dynamic range of D/M.

Although FIRE fundamentally presents a tradeoff in dynamic range forspeed, it improves in sensitivity when compared to single point scanningfluorescence microscopy, as multiplexing the sample excitation by afactor of M yields an M-fold increase in the dwell time of each pixel.However, due to the parallel nature of detection, FIRE shares shot noiseacross all pixels in a row. This causes the shot noise-limiteduncertainty at each pixel to scale with the square root of the totalnumber of photons collected from all pixels in a line scan. The extentto which this effect degrades the SNR at each pixel depends inversely onthe sparsity of the sample.

As described previously, direct digital synthesis of the localoscillator (LO) and radiofrequency (RF) comb beams can be accomplishedusing the two outputs of a 5 GS/s arbitrary waveform generator, forexample, which are amplified to drive the acousto-optic deflector (AOD)and acousto-optic frequency shifter (AOFS). After photodetection anddigitization, two digital de-multiplexing algorithms can be used torecover the fluorescence image from the frequency multiplexed data. Thefirst method to recover the fluorescent signal employs a short timeFourier transform (STFT). The second method uses an array of digitallock-in amplifiers to heterodyne and de-multiplex each comb-lineseparately.

The image reconstruction algorithm 100 of the first method is shownschematically in FIG. 5 . The STFT method works by segmenting the datasequence into frames, and performing a discrete Fourier transform (DFT)on each frame to recover the frequency-multiplexed signal. Each framecorresponds to a 1D line-scan. In order to avoid pixel-pixel cross talkfrom power spreading across frequency bins, the time duration of eachframe is set as an integer multiple of the inverse of the frequency combspacing. In this case, the frequency channels are said to be orthogonal,and the DFT bins lie precisely on the frequency comb lines. The maximumline rate in this scenario is equal to the frequency comb spacing andcorresponds to 300 kHz, 400 kHz, and 800 kHz line rates.

Accordingly, as seen in FIG. 5 , the signal input 102 is divided intoframes 106 and subject to a fast Fourier transform 108. At the sametime, the reference comb 104 is subject to a fast Fourier transform 110and the comb-line frequencies are identified and recorded at 112. Thesignal values at the comb-line frequencies are recorded at block 114 anda color-mapped image is then generated 116.

In the second image reconstruction method, there are two digital lock-inamplifier approaches that may be used that are illustrated in FIG. 6 andFIG. 7 respectively. Generally, in the second image reconstruction, alock-in amplifier demodulation technique is implemented by digitallymixing the data signal with copies of each frequency comb line. Themixing of the comb lines and the signal downshifts the correspondingcomb line to baseband. A low pass filter (LPF) can be used to extinguishall other comb lines, leaving only the modulated fluorescent signal fromthe comb line of interest. To obviate phase locking the reference to thesignal, both in-phase (I) and quadrature phase (Q) mixing terms may becalculated. The magnitude of the sum of I- and Q-channels is equal tothe amplitude of the signal. The bandwidth of the LPF is less than halfthe comb spacing to prevent pixel crosstalk in frequency space. With thereduced analog bandwidth after filtering, each pixel's signal can beboxcar-averaged and under-sampled to at least the Nyquist rate, equal tothe frequency comb spacing. The under-sampled data rate corresponds tothe line rate of the system.

Accordingly, the image reconstruction 120 in FIG. 6 , the signal 122 andreference comb 124 are provided as inputs. The reference comb 124 issubject to a fast Fourier transform 126 and the comb-line frequenciesare recorded at 128 and the individual comb-lines are filtered at 130.The filtered comb-lines 130 are subject to an inverse fast Fouriertransform 132 and the signal 122 is mixed with each comb-tone at 134.The mixed signal is filtered with a low pass filter 136 to determine thetemporal resolution that is undersampled at 138 to determine the linerate. A color mapped image is then formed 140 from the resulting signal.

An alternative embodiment of the lock-in amplifier demodulation imagereconstruction is shown in FIG. 7 . Here, the same reference combprocessing and initial signal processing takes place as shown in FIG. 6. However, each comb of the mixed signal/comb-tone output is filtered inparallel by a digital low pass filter 152. The filtered signal isoversampled and averaged at block 154 and a color-mapped image is formedat block 156.

Although both the DFT and lock-in techniques can be used to de-multiplexthe fluorescence image, the lock-in technique has certain advantages:(a) there is no orthogonality requirement, leaving more flexibility incomb line configuration, (b) the nominal line rate is determined by theunder-sampling factor, allowing for line-rates above the minimum Nyquistrate, and (c) the reference and signal can be phase locked, either by apriori estimation of the signal phase or via deduction from the ratio ofI and Q channels. Phase locked operation rejects out of phase noise,resulting in a 3-dB improvement in the signal to noise ratio (SNR).

The invention may be better understood with reference to theaccompanying examples, which are intended for purposes of illustrationonly and should not be construed as in any sense limiting the scope ofthe present invention as defined in the claims appended hereto.

EXAMPLE 1

In order demonstrate the operational principles of FIRE microscopy, asample of immobilized 15-μm diameter fluorescent polystyrene beads wereimaged using 256 excitation frequencies spaced by 300 kHz (totalbandwidth of 76.8 MHz). The images were collected at a frame rate of 4.2kHz. The detected time-domain PMT signal offers no indication of thelateral bead location. A fast Fourier transform (FFT) of three windowsof the signal was used to indicate the different frequency componentsassociated with the positions of each bead.

This illustrated the frequency-to-space mapping of the sample excitationand emission. The vertical location of the beads in the image wasrecovered from the reference output of the resonant scan mirror. Toavoid nonlinear space-to-time mapping from the resonant scan mirror inthe vertical direction, an aperture is placed in the intermediate imageplane to limit the sample excitation to the linear portion of the scanfield, and a sine correction algorithm is further applied to the imageto compensate for any residual distortion in the image.

The axial resolution of the FIRE system was measured by exciting a 500nm fluorescent bead with 488-nm excitation, and scanning the samplethrough the focus at 1 μm intervals, using 488-nm excitation. A 100×,1.4-NA oil immersion objective was used in combination with a 200-mmfocal length tube lens and a 100-μm tall slit aperture placed before thePMT. A Gaussian fit to the data showed a 5.9 μm FWHM. This axialresolution could be further improved through the use of a smaller slit,as the 100 μm slit is larger than the Airy disk at this magnification.

To demonstrate the ability of two-dimensional FIRE to record fluorescentdynamic phenomena, fluorescent beads flowing at a velocity of 2.3 mm s⁻¹inside a microfluidic channel were imaged. In the two-dimensionalimplementation of FIRE, images are acquired on an inverted microscope,in a de-scanned configuration, using a 2.2 kHz resonant scan mirror. Toavoid nonlinear space-to-time mapping from the resonant scanner in thevertical direction, an aperture was placed in the intermediate imageplane of the imaging system to limit the sample excitation to theapproximately linear portion of the scan field. A sine correctionalgorithm was further applied to the image in Matlab to compensate forany residual distortion in the image arising from the sinusoidaldeflection pattern of the mirror. Brightness and contrast adjustments,thresholding, and 2-dimensional low-pass filtering were performed on allimages.

Fluorescence emission was detected by a bialkali photomultiplier tube(PMT) (R3896, Hamamatsu) after passing through a dichroic mirror and abandpass filter. The current signal from the PMT was amplified by a400-MHz bandwidth current amplifier with 5 kV/A transimpedance gain.This voltage signal was digitized using a 250 MS/s, 8-bit resolutionoscilloscope. For 2-D scanned images, the reference output from theresonant scan mirror was used for triggering, and was also digitized andsaved for image reconstruction. For the flow cytometry experiments, thedigitizer was triggered off of the image signal itself. The digitizeddata was then transferred to a PC for signal processing.

EXAMPLE 2

To demonstrate FIRE microscopy on biological samples, adherent cellsstained with various fluorophores were imaged at a frame rate of 4.4kHz. NIH 3T3 mouse embryonic fibroblasts, C6 astrocyte rat glialfibroblasts and Saccharomyces cerevisiae yeast were stained with afluorescent cytosol stain (Calcein AM) or nucleic acid stain (Syto16).NIH 3T3 and MCF-7 cells and C6 astrocytes were propagated in Dulbecco'sModified Eagle Medium with 10% fetal bovine serum and 1% penicillinstreptomycin at 37° C. and 5% CO₂. Liquid cultures of Saccharomycescerevisiae were grown in tryptic soy broth at 240 rpm and 37° C.

Prior to staining, cultured mammalian cells were released from cultureflasks, seeded on glass slides, and allowed to spread for 24 hours.Mammalian cells were stained with either 4 μM Syto16 green fluorescentnucleic acid stain in phosphate buffered saline (PBS) for 30 minutes, 1μM Calcein Red-Orange AM in culture media for 45 minutes, or 1 μMCalcein AM in culture media for 45 minutes. Cells were washed twice withPBS then fixed for 10 minutes with 4% paraformaldehyde in PBS. Followingfixation, cells were washed twice with PBS and prepared for eitherstationary or flow-through microfluidic imaging.

For stationary imaging, number 1.5 cover glasses were placed on slidesand sealed. In an effort to preserve the shape of adhered mammaliancells for flow-through microfluidic experiments a cell scraper was usedto remove and spread the stained and fixed cells from glass slides. Thecells were diluted in PBS.

S. cerevisiae were stained in suspension using the same concentration ofCalcein AM for 45 minutes, washed twice with PBS, fixed with 4%paraformaldehyde in PBS, and washed twice with PBS. For stationaryimaging, the cells were seeded on glass slides and sealed under number1.5 coverslips.

To compare FIRE microscopy with wide-field fluorescence imaging, laserexcitation at 488 nm was used for FIRE imaging (8.5 mWper pixel,measured before the objective), and mercury lamp excitation was used forwide-field imaging. All FIRE images that were generated used aradiofrequency comb frequency spacing of 400 kHz, and were composed of200×92 pixels. Slight vignetting was observed in the FIRE images due tothe mismatch of the Gaussian profile of the LO beam with the flat-topradiofrequency comb beam. This mismatch and the resulting vignetting canbe eliminated using digital pre-equalization of the radiofrequency combin the direct digital synthesis generator.

Images of cells were taken with both the FIRE microscope and aconventional widefield fluorescence microscope based on a 1,280×1,024pixel CMOS camera. The frame rate difference of nearly three orders ofmagnitude is mitigated by the gain of the PMT detector combined withdigital lock-in amplifier image demodulation. Comparisons of singleframes indicated that the intensity per pixel of the FIRE excitation wasapproximately 10 times that of the wide-field excitation. However, thetotal excitation energy incident upon the sample per pixel per frame inthe FIRE images was 100 times less than in the wide-field images.

EXAMPLE 3

Flow cytometry is an application that requires high-speed fluorescencemeasurements. Compared to single-point flow cytometry, imaging flowcytometry provides information that can be particularly useful forhigh-throughput rare cell detection. The high flow velocities associatedwith flow cytometry demand fast imaging shutter speeds andhigh-sensitivity photodetection to generate high-SNR, blur-free images.Conventional imaging flow cytometers use time delay and integration CCDtechnology to circumvent this issue, but the serial pixel readoutstrategy of this technology currently limits devices to a throughput ofapproximately 5,000 cells per second.

To demonstrate high-speed imaging flow cytometry using FIRE, a singlestationary line scan of 125 pixels spaced by 800 kHz (pixel readout rateof 100 MHz, line scan rate of 800 kHz) was used to image Syto16-stainedMCF-7 human breast carcinoma cells flowing in a microfluidic channel ata velocity of 1 m s⁻¹. Assuming a cell diameter of 20 mm, this velocitycorresponds to a throughput of 50,000 cells per second.

For the imaging flow cytometry, MCF-7 breast carcinoma cells werestained with Syto16, prior to fixation with formaldehyde. The cells werethen suspended in phosphate-buffered saline, and flowed through alinear, rectangular cross-section 110 μm×60 μm microfluidic channel madefrom polydimethylsiloxane, using a syringe pump, at a fixed volumetricrate. The fluid flow velocity is calculated using the equation V=Q/Awhere Q is the volumetric flow rate, and A is the cross-sectional areaof the channel. Vertical scaling calibration of the images was performedafter imaging 10 μm spherical beads flowing at the same volumetric flowrate.

For comparison, Syto16-stained MCF-7 cells in a fluid flow at the samevelocity using a frame transfer EMCCD in single exposure mode (512 x 512pixels) were also imaged. Although the high sensitivity and gain of theEMCCD together yield a reasonable SNR image, the camera's minimumexposure time of 10 ms and its frame transfer nature create significantblur at these flow velocities. In contrast, the FIRE line scan shutterspeed of 1.25 ms yields blur-free images with comparable SNR. Thisconfiguration of the FIRE system features pixel readout rates in the 100MHz range, but this rate can be directly extended to more than 1 GHzthrough the use of wider bandwidth acousto-optic deflectors. FIRE'smaximum modulation frequency, and thus maximum pixel readout rate, isintrinsically limited by the sample's fluorescence lifetime.

It can be seen that the radiofrequency-multiplexed excitation approachof the method can be adapted to many different diagnostic devices. Forexample, the methods can be adapted to Fast laser scanning fluorescencemicroscopy used in neurology, physiology, immunology, research. Themethod can be adapted for use in multi-photon excited fluorescencemicroscopy as well as stimulated Raman, CARS, 2nd harmonic and 3rdharmonic microscopy, etc. Other imaging modalities such as fastfluorescence lifetime imaging microscopy (frequency domain FLIM),imaging flow cytometry using fluorescence and multi-flow channelsimultaneous single-point fluorescence-based flow cytometry can alsobenefit from the methods.

Although the description herein contains many details, these should notbe construed as limiting the scope of the invention but as merelyproviding illustrations of some of the presently preferred embodimentsof this invention. Therefore, it will be appreciated that the scope ofthe present invention fully encompasses other embodiments which maybecome obvious to those skilled in the art.

Embodiments of the present invention may be described with reference toflowchart illustrations of methods and systems according to embodimentsof the invention, and/or algorithms, formulae, or other computationaldepictions, which may also be implemented as computer program products.In this regard, each block or step of a flowchart, and combinations ofblocks (and/or steps) in a flowchart, algorithm, formula, orcomputational depiction can be implemented by various means, such ashardware, firmware, and/or software including one or more computerprogram instructions embodied in computer-readable program code logic.As will be appreciated, any such computer program instructions may beloaded onto a computer, including without limitation a general purposecomputer or special purpose computer, or other programmable processingapparatus to produce a machine, such that the computer programinstructions which execute on the computer or other programmableprocessing apparatus create means for implementing the functionsspecified in the block(s) of the flowchart(s).

Accordingly, blocks of the flowcharts, algorithms, formulae, orcomputational depictions support combinations of means for performingthe specified functions, combinations of steps for performing thespecified functions, and computer program instructions, such as embodiedin computer-readable program code logic means, for performing thespecified functions. It will also be understood that each block of theflowchart illustrations, algorithms, formulae, or computationaldepictions and combinations thereof described herein, can be implementedby special purpose hardware-based computer systems which perform thespecified functions or steps, or combinations of special purposehardware and computer-readable program code logic means.

Furthermore, these computer program instructions, such as embodied incomputer-readable program code logic, may also be stored in acomputer-readable memory that can direct a computer or otherprogrammable processing apparatus to function in a particular manner,such that the instructions stored in the computer-readable memoryproduce an article of manufacture including instruction means whichimplement the function specified in the block(s) of the flowchart(s).The computer program instructions may also be loaded onto a computer orother programmable processing apparatus to cause a series of operationalsteps to be performed on the computer or other programmable processingapparatus to produce a computer-implemented process such that theinstructions which execute on the computer or other programmableprocessing apparatus provide steps for implementing the functionsspecified in the block(s) of the flowchart(s), algorithm(s), formula(e),or computational depiction(s).

From the discussion above it will be appreciated that the invention canbe embodied in various ways, including the following:

1. An apparatus for optical interrogation of a sample, comprising (a) afirst beam generator, comprising (1) a laser light source; (2) anacousto-optic deflector; and (3) a radio frequency comb generator; (b) asecond beam generator, comprising: (1) a laser light source; (2) anacousto-optic frequency shifter; and (3) a radio frequency tonegenerator; (c) an objective lens system configured to expose a sample tobeams from the first and second beam generators over time; and (d) aphotodetector; wherein light emissions from the sample from exposure tobeams from the first and second beam generators are detected by thephotodetector.

2. An apparatus as recited in any previous embodiment, wherein theobjective lens system comprises: a scanning mirror; a scan lens; a tubelens; and an objective lens.

3. An apparatus as recited in any previous embodiment, wherein thephotodetector comprises: a fluorescence emission filter; a slitaperture; and a photo-multiplier tube.

4. An apparatus as recited in any previous embodiment, the photodetectorfurther comprising: a computer with programming for generating an imageof the sample from detected fluorescence emissions.

5. An apparatus as recited in any previous embodiment, furthercomprising: a non-polarizing beam splitter configured to combine a beamfrom the first beam generator with a beam from the second beam generatorto produce a combined beam that is directed through the objective lenssystem to a sample.

6. An apparatus as recited in any previous embodiment, wherein the RFcomb generator comprises a direct digital synthesizer (DDS) RF combgenerator and said RF tone generator comprises a direct digitalsynthesizer (DDS) RF tone generator.

7. An apparatus as recited in any previous embodiment, furthercomprising: a controller configured to control the objective lens systemand the photodectector.

8. An apparatus for fluorescence imaging, comprising: (a) a laserexcitation source; (b) an acousto-optic interferometer for splittinglight from the laser excitation source into first and second frequencyshifted beams; (c) a beam splitter and lens system for combining thefrequency shifted beams and focusing the combined beams on a sample;(d)a photodetector for simultaneously detecting fluorescence emissions fromall points on the sample; and (e) a computer coupled to thephotodetector with programming for generating an image of the samplefrom the detected fluorescence emissions.

9. An apparatus as recited in any previous embodiment, wherein the firstfrequency shifted beam of the acousto-optic interferometer is producedwith a direct digital synthesizer (DDS) RF comb generator and anacousto-optic deflector.

10. An apparatus as recited in any previous embodiment, wherein thesecond frequency shifted beam of the acousto-optic interferometer isproduced with a direct digital synthesizer (DDS) RF tone generator andan acousto-optic frequency shifter.

11. An apparatus as recited in any previous embodiment, wherein thephotodetector comprises: a dichroic mirror; a fluorescence emissionfilter; a slit aperture; and a photo-multiplier tube.

12. A method of fluorescence imaging, the method comprising: (a)creating a first beam of frequency shifted light; (b) creating a secondbeam of frequency shifted light; (c) interrogating a sample with thefirst and second beams of light; and (d) detecting a response in thesample from exposure to the beams of frequency shifted light.

13. A method as recited in any previous embodiment, further comprising:combining the first beam and the second beam to produce a combined beam;and interrogating the sample with the combined beam.

14. A method as recited in any previous embodiment, wherein the firstbeam is created with an acousto-optic deflector driven by an RF comb andan excitation laser beam.

15. A method as recited in any previous embodiment, wherein the secondbeam is created with an acousto-optic frequency shifter driven by an RFtone and an excitation laser beam.

16. A method as recited in any previous embodiment, further comprising:assigning a distinct radiofrequency to each pixel in a row of pixels.

17. A method as recited in any previous embodiment, further comprising:generating an image of the sample from the detected fluorescenceemissions.

18. A method as recited in any previous embodiment, wherein the imagegeneration comprises: applying a Fast Fourier Transform to a referencecomb to identify comb-line frequencies; dividing a received signal intoframes; applying a Fast Fourier Transform to the divided signal;recording values of transformed signal at comb-line frequencies; andforming an image from the recorded values.

19. A method as recited in any previous embodiment, wherein the imagegeneration comprises: obtaining a signal from the detector; calculatingcomb tones; mixing the signal with each comb tone to produce a mixedsignal; passing the signal through a low pass filter to determinetemporal resolution; undersampling the filtered signal; and forming animage.

20. A method as recited in any previous embodiment, wherein the imagegeneration comprises: obtaining a signal from the detector; calculatingcomb tones; mixing the signal with each comb tone to produce a mixedsignal; passing each comb tone of the signal through a digital low passfilter in parallel; oversampling the filtered signals; and forming animage.

Although the description above contains many details, these should notbe construed as limiting the scope of the invention but as merelyproviding illustrations of some of the presently preferred embodimentsof this invention. Therefore, it will be appreciated that the scope ofthe present invention fully encompasses other embodiments which maybecome obvious to those skilled in the art, and that the scope of thepresent invention is accordingly to be limited by nothing other than theappended claims, in which reference to an element in the singular is notintended to mean “one and only one” unless explicitly so stated, butrather “one or more.” All structural, chemical, and functionalequivalents to the elements of the above-described preferred embodimentthat are known to those of ordinary skill in the art are expresslyincorporated herein by reference and are intended to be encompassed bythe present claims. Moreover, it is not necessary for a device or methodto address each and every problem sought to be solved by the presentinvention, for it to be encompassed by the present claims. Furthermore,no element, component, or method step in the present disclosure isintended to be dedicated to the public regardless of whether theelement, component, or method step is explicitly recited in the claims.No claim element herein is to be construed as a “means plus function”element unless the element is expressly recited using the phrase “meansfor”. No claim element herein is to be construed as a “step plusfunction” element unless the element is expressly recited using thephrase “step for”.

1-20. (canceled)
 21. An apparatus comprising: a light beam generatorconfigured to generate a plurality of frequency shifted beams of light;an optical apparatus configured to combine at least a first frequencyshifted beam of light with a second frequency shifted beam of light andcause interference between the first frequency shifted beam of light andthe second frequency shifted beam of light; and a photodetectorconfigured to detect light from a sample irradiated by the combinedinterfering beams of light.
 22. The apparatus according to claim 21,wherein the light beam generator comprises a laser.
 23. The apparatusaccording to claim 21, wherein the light beam generator comprises: afirst acousto-optic device for generating the first frequency shiftedbeam of light; and a second acousto-optic device for generating thesecond frequency shifted beam of light.
 24. The apparatus according toclaim 23, wherein the first acousto-optic device comprises anacousto-optic deflector (AOD).
 25. The apparatus according to claim 24,wherein the first frequency shifted beam of light is generated byapplying a comb signal to the acousto-optic deflector.
 26. The apparatusaccording to claim 24, wherein the second acousto-optic device comprisesan acousto-optic frequency shifter (AOFS).
 27. The apparatus accordingto claim 26, further comprising a beam splitter configured to propagatea first beam of light to the acousto-optic deflector and a second beamof light to the acousto-optic frequency shifter.
 28. The apparatusaccording to claim 21, further comprising a scanning component.
 29. Theapparatus according to claim 28, wherein the scanning componentcomprises a resonant scan galvanometer mirror configured to propagatethe combined beams of light onto the sample.
 30. The apparatus accordingto claim 21, further comprising a flow cell for propagating the samplethrough a flow stream.
 31. The apparatus according to claim 21, whereinthe photodetector is configured to detect fluorescence from the sample.32. A method comprising: generating a plurality of frequency shiftedbeams of light with a light beam generator; combining with an opticalapparatus at least a first frequency shifted beam of light with a secondfrequency shifted beam of light and causing interference between thefirst frequency shifted beam of light and the second frequency shiftedbeam of light; irradiating a sample with the combined interfering beamsof light; and detecting light from the irradiated sample with aphotodetector.
 33. The method according to claim 32, wherein the firstfrequency shifted beam of light is generated with a first acousto-opticdevice and the second frequency shifted beam of light is generated witha second acousto-optic device.
 34. The method according to claim 33,wherein the first acousto-optic device comprises an acousto-opticdeflector (AOD) and the second acousto-optic device comprises anacousto-optic frequency shifter (AOFS).
 35. The method according toclaim 34, wherein the method comprises applying a comb signal to theacousto-optic deflector.
 36. The method according to claim 32, whereinthe method comprises scanning the combined beams of light on the sample.37. The method according to claim 32, wherein the method comprisesdetecting fluorescence from the sample.